Deep learning-based full-reference image quality assessment (FR-IQA) models typically rely on the feature distance between the reference and distorted images. However, the underlying assumption of these models that the distance in the deep feature domain could quantify the quality degradation does not scientifically align with the invariant texture perception, especially when the images are generated artificially by neural networks. In this paper, we bring a radical shift in inferring the quality with learned features and propose the Deep Image Dependency (DID) based FR-IQA model. The feature dependency facilitates the comparisons of deep learning features in a high-order manner with Brownian distance covariance, which is characterized by the joint distribution of the features from reference and test images, as well as their marginal distributions. This enables the quantification of the feature dependency against nonlinear transformation, which is far beyond the computation of the numerical errors in the feature space. Experiments on image quality prediction, texture image similarity, and geometric invariance validate the superior performance of our proposed measure.
translated by 谷歌翻译
基于深度学习的图像质量评估(IQA)模型通常会学会从单个数据集中预测图像质量,从而导致该模型过度适合特定的场景。为此,混合的数据集培训可以是增强模型概括能力的有效方法。但是,将不同的iQA数据集组合在一起是无聊的,因为它们的质量评估标准,评分范围,视图条件以及在图像质量注释期间通常不共享主题。在本文中,我们没有对注释对准注释,而是为IQA模型学习提供了一个单调的神经网络,其中包括不同的数据集。特别是,我们的模型由数据集共享的质量回归器和几个特定于数据集的质量变压器组成。质量回归器旨在获得每个数据集的感知质量,而每个质量变压器则将感知质量映射到相应的数据集注释及其单调性。实验结果验证了提出的学习策略的有效性,我们的代码可在https://github.com/fzp0424/monotoniciqa上获得。
translated by 谷歌翻译
需要高质量的面部图像来保证在监视和安全场景中自动识别系统(FR)系统的稳定性和可靠性。但是,由于传输或存储的限制,在分析之前,通常会压缩大量的面部数据。压缩图像可能会失去强大的身份信息,从而导致FR系统的性能降低。在此,我们首次尝试研究FR系统的明显差异(JND),可以将其定义为FR系统无法注意到的最大失真。更具体地说,我们建立了一个JND数据集,其中包括3530个原始图像和137,670个由高级参考编码/解码软件生成的压缩图像,该图像基于多功能视频编码(VVC)标准(VTM-15.0)。随后,我们开发了一种新型的JND预测模型,以直接推断FR系统的JND图像。特别是,为了最大程度地删除冗余性,在不损害鲁棒身份信息的情况下,我们将编码器应用于多个功能提取和基于注意力的特征分解模块,以将面部特征逐渐分解为两个不相关的组件,即身份和残差特征,通过自我 - 监督学习。然后,剩余特征被馈入解码器以生成残差图。最后,通过从原始图像中减去残差图来获得预测的JND映射。实验结果表明,与最先进的JND模型相比,所提出的模型可以实现JND MAP预测的更高准确性,并且能够在维持FR系统的性能的同时保存更多的位置,而与VTM-15.0相比。
translated by 谷歌翻译
自然图像的统计规律(称为自然场景统计数据)在不引用图像质量评估中起重要作用。但是,人们普遍认为,通常是计算机生成的屏幕内容图像(SCI)不持有此类统计信息。在这里,我们首次尝试学习SCI的统计数据,基于可以有效确定SCI的质量。所提出的方法的基本机制是基于一个狂野的假设,即没有物理上获得的SCI仍然遵守某些可以以学习方式理解的统计数据。我们从经验上表明,在质量评估中可以有效利用统计偏差,并且在不同的环境中进行评估时,提出的方法优越。广泛的实验结果表明,与现有的NR-IQA模型相比,基于深度统计的SCI质量评估(DFSS-IQA)模型可提供有希望的性能,并在跨数据库设置中显示出很高的概括能力。我们的方法的实现可在https://github.com/baoliang93/dfss-iqa上公开获得。
translated by 谷歌翻译
现有的基于深度学习的全参考IQA(FR-IQA)模型通常通过明确比较特征,以确定性的方式预测图像质量,从而衡量图像严重扭曲的图像是多远,相应的功能与参考的空间相对远。图片。本文中,我们从不同的角度看这个问题,并提议从统计分布的角度对知觉空间中的质量降解进行建模。因此,根据深度特征域中的Wasserstein距离来测量质量。更具体地说,根据执行最终质量评分,测量了预训练VGG网络的每个阶段的1Dwasserstein距离。 Deep Wasserstein距离(DEEPWSD)在神经网络的功能上执行的,可以更好地解释由各种扭曲引起的质量污染,并提出了高级质量预测能力。广泛的实验和理论分析表明,在质量预测和优化方面,提出的DEEPWSD的优越性。
translated by 谷歌翻译
在本文中,我们提出了通过特征级伪参考(PR)幻觉提出的无引用(NR)图像质量评估(IQA)方法。提出的质量评估框架基于自然图像统计行为的先前模型,并植根于以下观点,即可以很好地利用具有感知意义的特征来表征视觉质量。本文中,通过以原始参考为监督的相互学习方案学习了扭曲的图像中的PR特征,并通过三重态约束进一步确保PR特征的区分特性。给定质量推断的扭曲图像,特征水平的分离是用可逆神经层进行最终质量预测的,导致PR和相应的失真特征以进行比较。在四个流行的IQA数据库中证明了我们提出的方法的有效性,跨数据库评估的卓越性能也揭示了我们方法的高概括能力。我们的方法的实现可在https://github.com/baoliang93/fpr上公开获得。
translated by 谷歌翻译
Deep learning models can achieve high accuracy when trained on large amounts of labeled data. However, real-world scenarios often involve several challenges: Training data may become available in installments, may originate from multiple different domains, and may not contain labels for training. Certain settings, for instance medical applications, often involve further restrictions that prohibit retention of previously seen data due to privacy regulations. In this work, to address such challenges, we study unsupervised segmentation in continual learning scenarios that involve domain shift. To that end, we introduce GarDA (Generative Appearance Replay for continual Domain Adaptation), a generative-replay based approach that can adapt a segmentation model sequentially to new domains with unlabeled data. In contrast to single-step unsupervised domain adaptation (UDA), continual adaptation to a sequence of domains enables leveraging and consolidation of information from multiple domains. Unlike previous approaches in incremental UDA, our method does not require access to previously seen data, making it applicable in many practical scenarios. We evaluate GarDA on two datasets with different organs and modalities, where it substantially outperforms existing techniques.
translated by 谷歌翻译
The development of social media user stance detection and bot detection methods rely heavily on large-scale and high-quality benchmarks. However, in addition to low annotation quality, existing benchmarks generally have incomplete user relationships, suppressing graph-based account detection research. To address these issues, we propose a Multi-Relational Graph-Based Twitter Account Detection Benchmark (MGTAB), the first standardized graph-based benchmark for account detection. To our knowledge, MGTAB was built based on the largest original data in the field, with over 1.55 million users and 130 million tweets. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. In MGTAB, we extracted the 20 user property features with the greatest information gain and user tweet features as the user features. In addition, we performed a thorough evaluation of MGTAB and other public datasets. Our experiments found that graph-based approaches are generally more effective than feature-based approaches and perform better when introducing multiple relations. By analyzing experiment results, we identify effective approaches for account detection and provide potential future research directions in this field. Our benchmark and standardized evaluation procedures are freely available at: https://github.com/GraphDetec/MGTAB.
translated by 谷歌翻译
As one of the prevalent methods to achieve automation systems, Imitation Learning (IL) presents a promising performance in a wide range of domains. However, despite the considerable improvement in policy performance, the corresponding research on the explainability of IL models is still limited. Inspired by the recent approaches in explainable artificial intelligence methods, we proposed a model-agnostic explaining framework for IL models called R2RISE. R2RISE aims to explain the overall policy performance with respect to the frames in demonstrations. It iteratively retrains the black-box IL model from the randomized masked demonstrations and uses the conventional evaluation outcome environment returns as the coefficient to build an importance map. We also conducted experiments to investigate three major questions concerning frames' importance equality, the effectiveness of the importance map, and connections between importance maps from different IL models. The result shows that R2RISE successfully distinguishes important frames from the demonstrations.
translated by 谷歌翻译
Compressed videos often exhibit visually annoying artifacts, known as Perceivable Encoding Artifacts (PEAs), which dramatically degrade video visual quality. Subjective and objective measures capable of identifying and quantifying various types of PEAs are critical in improving visual quality. In this paper, we investigate the influence of four spatial PEAs (i.e. blurring, blocking, bleeding, and ringing) and two temporal PEAs (i.e. flickering and floating) on video quality. For spatial artifacts, we propose a visual saliency model with a low computational cost and higher consistency with human visual perception. In terms of temporal artifacts, self-attention based TimeSFormer is improved to detect temporal artifacts. Based on the six types of PEAs, a quality metric called Saliency-Aware Spatio-Temporal Artifacts Measurement (SSTAM) is proposed. Experimental results demonstrate that the proposed method outperforms state-of-the-art metrics. We believe that SSTAM will be beneficial for optimizing video coding techniques.
translated by 谷歌翻译